Method for operating an industrial robot

ABSTRACT

A method for operating an industrial robot by means of an operating device, including a step of touching a virtual operating element on a touch-sensitive display of a graphical user interface, said display being surrounded by a frame, wherein a function associated with the operating element is triggered when the virtual operating element is touched. In order to increase the safety of the operation of the industrial robot and to be able to adjust the position of the industrial robot in the desired range, the speed or distance of a deflection of a finger of an operating person during the touching of the virtual operating element of the touch display is detected.

The invention relates to a method for operating an industrial robot byan operating device comprising the method step of touching a virtualoperating element on a touch-sensitive display of a graphic operatorinterface surrounded by a frame, wherein when the virtual operatingelement is touched, a function associated with the operating element isinitiated.

A device for controlling an industrial robot in the form of a manualoperating device is described in DE 10 2010 039 540 A1. The manualoperating device can be coupled to a robot control in order to programor control the industrial robot.

The manual operating device comprises an electronic system with amicroprocessor in order to be able to communicate with the robotcontrol. Furthermore, the manual operating device comprises a displaydesigned as a touchscreen, an emergency stop key and a switchconstructed as a lock. In order to manually move, e.g., a robotic armthe manual operating device comprises input means or moving means thatcan be manually activated independently of each other and areconstructed, e.g., as a 6D mouse or as typing keys. There is thepossibility of associating each of the moving means with its ownreference coordinate system by the touchscreen.

However, in the known design the control of the industrial robot takesplace exclusively via the manually actuatable input means so that themanual operating device is expensive to produce and temperamental inoperation.

Another device for operating an industrial robot is described in DE 102010 025 781 A1. The manual device in the form of a mobile telephonecomprises a touchscreen that on the one hand serves as output means foroutputting information from the robot control, in particular forrepresenting an operating surface and at the same time serves as commandinput means for inputting control commands via keys.

The manual device is detachably fastened to a portable safety device bya clamping device such as is known in principle, for example, forholding mobile telephones in motor vehicles and is connected to it by aUSB interface. The safety input device comprises an emergency stopbutton, an enabling switch and a selector switch for the operating type.A disadvantage of this embodiment is that an operator is always forcedin order to safely operate the virtual keys to look at the touchscreenin order to avoid false inputs. The same applies in the case of poorenvironmental conditions such as, for example, a strong incidence oflight or darkness that would made an operating of the touchscreen moredifficult.

The literature passage, the Ph.D. thesis of Mark John Micire:“Multi-Touch Interaction for robot Command and Control”, December 2010describes the actuation of a robot by moving a finger on a touch screen.The direction in a plane and the speed of a robot can be controlled bythe movement of the finger. When the finger is removed from the touchscreen the robot comes to a standstill.

According to US 2008/0165255 a vehicle can be controlled in twodimensions by the movement of a finger on the surface of a touchscreen.

A multi-touchscreen is described in the literature passage NIEDERHUBER,Ch.: “Bedienpanel: Multitouch-Technologie mit Haptischen Elementsvereint” [German=Operating Panel: Multi-Touch-Technology Combined withHaptic Elements]”, in: etz Elektrotechnik+Automation, issue 1-2/2012,pp. 2-4.

The present invention is based on the problem of further developing amethod of the initially cited type in such a manner that the safety isincreased when operating an industrial robot. Also, the industrial robotshould be moved in its position to the desired extent. A controlling ofthe robot in its direction of movement and its speed should be possible.

In order to solve the problem it is suggested that the speed or thestretch of a deflection of a finger of an operating person when touchingthe virtual operating element of the touch display constructed as adefined moving surface is selectively detected, that a control signal isgenerated from the speed or the stretch of the deflection that is usedto set the speed for the position for the movements of the industrialrobot, and that a first moving surface for a two-dimensional setting ofposition (X-, Y- coordinates (position) or V_(x), V_(y) speed) isdefined at a distance from a frame section serving as a haptic mark, andthat a second moving surface for a third setting of position (Zcoordinate (position) or V_(z) speed) of the industrial robot is definedinside the distance.

In particular, a control signal is generated by the deflection of thefinger during the touching of one of the moving surfaces which signal isused for setting the speed or the position for movements of theindustrial robot as a function of the adjusted mode of the operatingdevice.

According to the invention a three-dimensional movement of a robot ismade possible by a 2-D movement on a touchscreen. At this time a guidingof a finger such as the thumb along a section of the frame surroundingthe touch display and serving as a haptic mark takes place in order toadjust the robot in a direction. An adjustment in the remainingdirections of the Cartesian coordinate system or of a defined coordinatesystem takes place at a distance to the finger such as the thumb. As aconsequence of the special arrangement the operator can clearlydistinguish and blindly reach the two moving fields. The field directlyon the display edge—the finger has perceptible contact with the housingedge—will actuate the setting of the movement for the 3d dimension. Thefield about one finger width next to the display edge actuates thesetting of movement in the remaining two dimensions. The finger such asthe thumb is distinctly spread to the side here and has no contact withthe housing edge. If the operator wants to decide for the first or thesecond virtual moving surface, the accuracy of the finger is ofsecondary importance. The first finger contact with the touch displayonly has to be inside the generously dimensioned one or the othervirtual moving surface. The first striking point of the finger is takenas the zero point from which each other finger movement—under permanentfinger contact with the touch display—is evaluated as a deflection fromwhich a control signal for the setting of the movement of the robot isgenerated. This movement supported by the haptic mark of the housingedge permits the movement of the robot in blind operation.

There is furthermore the possibility of switching by changing the modebetween the setting of position and the setting of speed for the robot,wherein the same moving surfaces of the touch display can be used in thepreviously described manner.

The stretch of the deflection of the finger on the defined movingsurface or virtual operating element is preferably proportional to themoving speed and/or the position of the industrial robot. After thecontacting of the defined moving surface the entire touch displaysurface, even beyond the field boundaries of the virtual moving surfaceshown, is available to the finger during the contacting sliding.

The industrial robot comes to a standstill when the operating personlifts his finger from the touch display. Furthermore, it is providedthat the sensitivity of the reaction to the movement of a finger overvirtual operating element such as a slide regulator is continuouslyadjusted for the setting of the position as well as for the setting ofthe speed.

A two-dimensional setting of position such as X and Y coordinatespreferably takes place by touching a defined moving surface that ispositioned in such a manner on the touch display at a distance to thedisplay edge that the moving surface can be actuated with a finger suchas the thumb.

The area of the touch display, which can be a commercially availabletouchscreen with a smooth surface and that is preferably constructed asa capacitive touch screen, even though a resistive touch screen can alsobe considered in which a finger, in particular the thumb, must be movedby an operating person in a contacting manner on the touch display inorder to set the X-Y coordinates of the robot, runs in a so-callededge-remote area of the touch display, preferably at a distance of atleast one thumb-width to the frame surrounding the touch display.

The invention furthermore provides that the industrial robot can becontrolled via an operating element in a third coordinate direction (Zcoordinate) and arranged laterally on the display edge, wherein thefinger of an operating person is guided by the display edge and/or onthe display edge and along a haptic mark running from the latter.

The X, Y and Z coordinates are in particular coordinates of theCartesian coordinate system. The tool orientations (angles of rotation)A, B and C can be controlled by adding at least one other or othervirtual moving surfaces. These other moving surfaces are comparable inconstruction and function to the moving surfaces for the X, Y and Zmoving settings and are represented in common with them. The placing onthe touch display is selected in such a manner that the moving surfacefor the moving direction “C” borders on an edge section of the housingframe (the latter serves as a haptic mark) and the moving surface forthe moving directions “A” and “B” are placed at a defined distance fromthe housing edge.

Therefore, all 6 degrees of freedom of a robot can be controlled inblind operation with one touch display.

If a movement is not to take place but rather exclusively an influencingof speed in the X, Y and/or Z direction or A, B and/or C direction, themode is switched by touching an appropriate virtual element on the touchdisplay in order to adjust the moving speed in the previously describedmanner.

In other words, the possibility is utilized after the touching of thetouch display to use the finger more or less to “pull” in order toproduce an analogous movement setting for the robot. Therefore, therobot can be sensitively controlled in 6 degrees of freedom (e.g., X, Y,Z and tool orientation A, B, C).

It is possible with the deflection of the finger to give a setting for aposition to the robot similar to the cursor control by touchpad in thecase of a notebook. The robot can be simultaneously moved intocoordinate directions, e.g., X and Y and A and B.

In another mode a speed setting is generated for the robot by the fingerdeflection: the more the finger is deflected the quicker the robotmoves.

After touching the selected moving surface, the finger can subsequentlyalso be drawn over the over the field boundary over the entire display,therefore producing movement settings. After letting loose, the robotremains immediately standing. Then, the desired field must be hit againfor another movement setting.

The sensitivity of the reaction to the finger movement can becontinuously adjusted by a regulator, e.g., the override, for theposition setting and also for the speed setting.

The sensitive surface for the 2-D moving (X, Y direction and/or A, Bdirection) is placed in the vicinity of the display edge so that it canstill be reached quite well with the finger (e.g. with the thumb spreadout) at a distinct distance from the display edge.

In order to also be able to move in the third coordinate direction (Zdirection or C direction) a finger-wide field is placed directly on theside of the display edge so that this field can be touched with thefinger. This field generates a one-dimensional moving setting (Zdirection or C direction).

The operator can clearly distinguish and blindly reach the two movingfields by the special arrangement: The field directly on the displayedge (finger has perceptible contact with the housing edge) actuates themoving setting for the third dimension. The field that is approximatelyone finger-width wide adjacent to the display edge actuates thesimultaneous moving setting in two dimensions.

An industrial robot has 6 degrees of freedom. For the adjusting of thetool orientation with the three angles (A, B, C) the same method asdescribed above is used. Here the display is divided into two zones. Inone zone (e.g., the upper one) the fields for the dimensions 1-3 (e.g.,X, Y, Z) are located. In the other zone (e.g., the lower one) the movingfields for the dimensions 4-6 (e.g., A, B, C) are located. Both zonescan be distinguished blindly with haptic marks.

A multi-touch display or single-touch display is used as touch display.

With a multi-touch display the robot can therefore be movedsimultaneously in all 6 degrees of freedom. With a single-touch displaythe moving functions can only be used sequentially.

The device in accordance with the invention is distinguished over theprior art in particular in that the number of hardware components isreduced to an absolute minimum. All operating functions are consequentlyrealized in touch software with the exception of the safety-relatedswitch “emergency stop” and “enabling”. No other electrical componentssuch as foil keys, switches or signal lights are necessary. As aconsequence, the system is low-maintenance.

The space gained favors a large, comfortable touch display. The virtualoperating elements and displays shown on the touch display are designedfor industrial use and are shown contrast-rich and large so that areliable operation is possible.

Another preferred embodiment of the method provides that the touching ofthe virtual operating element on the surface of the touch display isdetected by determining a first coordinate of a contact point, and thatan initiation of the function of the virtual operating element takesplace when the first coordinate of the contact point leaves, after aconstant contact with the surface of the touch display, a givencoordinate area by a manual action of the operating person.

The manual action can be initiated by a gesture of the operating person.The gesture can be performed by drawing a finger of the operating personon the touch display into or out of the given coordinate area. Thegesture is preferably carried out in a defined direction, wherein thesensitivity of the reaction to the finger movement, the intensity of thegesture, can be continuously adjusted.

Furthermore, the invention refers to a characteristic inventive methodfor operating an industrial robot with a manual device. It provides thatthe initiation of a touch function requires a manual action of theoperator on the touch display. In order to prevent an unintendedinitiation of virtual operating elements by an unintended touching, afunction is not initiated until after the touching of the touch displaya special “small movement” is made, e.g., drawing the finger in adefined direction. This results in a reliable touch.

The gesture required for initiating a function, that is, the requiredintensity or type of gesture, can be continuously adjusted: It extendsfrom a simple finger contact, generally customary operation of the touchdisplay, to a defined gesture. The finger can slide in continuation ofthe finger troughs on the touch display by haptic marks such as thespecial impressing of finger troughs in the display edge and initiate afunction at this time. If the operator notices that he initiated anundesired initiation of a function he can suppress the initiation of thefunction by withdrawing his finger into the original position.

Other details, advantages and features of the invention result not onlyfrom the claims, the features to be gathered from them—alone and/or incombination—, but also from the following description of exemplaryembodiments to be gathered from the drawings.

In the drawings:

FIG. 1 shows a programmable manual device for operating an industrialrobot,

FIG. 2 shows a section of a display frame of the programmable manualdevice which frame is adjacent to the touch display,

FIG. 3 shows a second section of a display frame adjacent to the touchdisplay, and

FIG. 4 shows a third section of a display frame adjacent to the touchdisplay,

FIG. 5 shows a rear view of the programmable manual device,

FIG. 6 shows a schematic method of operating an industrial robot with anoperating device,

FIG. 7 shows a section of an operating surface of the operating devicewith virtual operating elements,

FIG. 8 shows a section of an operating surface of the operating devicewith an image with graphical information,

FIG. 9 shows an operating surface of the operating device with differentmoving surface, and

FIG. 10 shows a virtual operating element for recalibrating thecoordinate system of the operating device.

FIG. 1 shows a device 10 in the form of a programmable manual device foroperating an industrial robot 12. To this end the manual device 10 isconnected by a wireless or wired communication connection 14 to a robotcontrol 16. The manual device 10 comprises a graphical operatorinterface 18 for the touch-sensitive display 20, called touch display inthe following. The touch display 20 serves to display at least onevirtual operating element 22.1 . . . 22.n, 24.1 . . . 24.n thatrepresents a function for controlling, programming or operating theindustrial robot 12, wherein when the virtual operating element 22.1 . .. 22.n, 24.1 . . . 24.n is touched with a finger of an operator or by apen, the associated function is initiated.

The manual device 10 furthermore comprises a control unit 30 forcontrolling the graphical operator interface 18 and for communicatingwith the robot control 16 as well as a position sensor for determiningposition and inclination of the operating device.

The graphical operator interface 18 with the touch display 20 isarranged together with the control unit 30 in a housing 32. The housing32 forms a display frame 34 that surrounds the touch display 20 on theedges. In addition, a safety-relevant “emergency stop” switch 26 isarranged on an upper side of the housing 32.

The virtual operating elements 22.1 . . . 22.n and 24.1 . . . 24.n arearranged along a frame section 36, 38 of the display frame and adjacentto the touch display 20. In order to make possible a blind operation ofthe virtual operating elements 22.1 . . . 22.n and 24.1 . . . 24.n,according to a first characteristic concept of the invention hapticmarks 40.1 . . . 40.n and 42.1 . . . 42.n are arranged in the framesections 36, 38. Each haptic mark 40.1 . . . 40.n, 42.1 . . . 42.n canbe associated with a virtual operating element 22.1 . . . 22.n, 24.1 . .. 24.n.

In particular, the virtual operating element 22.1 . . . 22.n, 24.1 . . .24.n is directly adjacent to the haptic mark 40.1 . . . 40.n and 42.1 .. . 42.n, so that a direct transition from the haptic mark 40.1 . . .40.n and 42.1 . . . 42.n to the virtual operating element 22.1 . . .22.n, 24.1 . . . 24.n takes place. Consequently, a finger guided along ahaptic mark 40.1 . . . 40.n, or 42.1 . . . 42.n is guided quasi in onestroke to the virtual operating element 22.1 . . . 22.n, 24.1 . . .24.n. This avoids or minimizes erroneous operations: At first, theposition of the virtual operating element is sensed with the aid of thehaptic mark and subsequently the function is initiated by touching thevirtual operating element. Furthermore, it is not necessary that thetouch display 20 has to have a special shape. In particular, and indeviation to the prior art, it is not necessary that special, overlaidmaterials are applied onto the touch display as a result of which lossesof transparency would occur.

The haptic marks 40.1 . . . 40.n and 42.1 . . . 42.n form a guide bywhich a finger of an operating person is guided to the associatedvirtual operating element 22.1 . . . 22.n, 24.1 . . . 24.n.

FIG. 2 shows an enlarged review of the operating elements 22.1 . . .22.n and the marks 40.1 . . . 40.n associated with them.

The arranging of the virtual operating elements 22.1 . . . 22.n and 24.1. . . 24.n along the frame sections 36, 38 comprising the haptic marks40.1 . . . 40.n, 42.1 . . . 42.n ensures a reliable operation of thevirtual operating elements. The recessed and specially defined framesections 36, 38 serve for the tactile orientation on the touch display20.

In the exemplary embodiment shown in FIG. 2 the haptic marks 40.1 . . .40.n, 42.1 . . . 42.n are designed as finger troughs that are formed insuch a manner that they can be reliably sensed with the fingers andensure a guiding of the finger from the frame sections 36, 38 in thedirection of the associated virtual operating element 22.1 . . . 22.nand 24.1 . . . 24.n.

Furthermore, haptic marks 43.1 . . . 43.n are provided that areconstructed as nubs and are arranged on a surface of the display frame34.

This compensates on the one hand the main lacking haptic of a touchdisplay and on the other hand operator can direct his visual attentiononto the industrial robot in the process without having to look at themanual operating device, as a result of which the operating safety isincreased on the whole. A “blind operation” is made possible.

FIG. 3 shows an embodiment of a haptic mark 44 as a frame corner 46 ofthe display frame 34 bordering on the touch display 20. An unambiguous,exact position on the touch display 20 is defined by the frame corner 46of the display frame 34. A virtual operating element 48 such as avirtual slide element is provided on the touch display 20 at theseselected positions that is moved for example, in a linear directionalong a display-side frame section 50 or along another frame section 52of the frame corner 46. The view and the function of the virtual slideelement 48 changes as a function of the selected direction of movement.

FIG. 4 shows another embodiment of a haptic mark 54 designed as adisplay-side frame section 56 of the display frame 34. The finger of anoperating person can execute a shifting movement along the frame section56 by which a virtual slide element 60 running along the frame section56 can be adjusted.

The haptic marks 40.1 . . . 40.n, 42.1 . . . 42.n shown in FIGS. 1 and 2and constructed as finger troughs form a haptic orientation of thedisplay edge with high resolution, e.g., for the sensing of positions ofthe virtual operating elements 22.1 . . . 22.n, 24.1 . . . 24.n sincethey are arranged directly adjacent to the finger troughs. Each fingertrough can be unambiguously associated with the virtual operatingelement. The finger troughs 22.1 . . . 22.n, 24.1 . . . 24.n areconstructed to be half-open and opened in the direction of the touchdisplay 20 so that a finger can slide as guided as in a groove on to thetouch display 20 and can initiate a function of the virtual operatingelement 22.1 . . . 22.n, 24.1 . . . 24.n there.

According to an independent concept of the invention it is provided thatthe initiation of a function associated with the virtual operatingelement 22.1 . . . 22.n, 24.1 . . . 24.n requires a manual action of theoperating person on the touch display 20. In order to prevent anunintended initiation of the virtual operating elements 22.1 . . . 22.n,24.1 . . . 24.n by an unintended touching, a function is not initiateduntil a pre-defined gesture such as, for example, moving a finger in adefined direction is carried out after the touching of the virtualoperating element on the touch display 20. The sensitivity of thereaction to the finger movement can be gradually adjusted by aregulator. Therefore, the intensity of the required gesture forinitiating functions can be gradually adjusted. It extends from a simplefinger contact, a generally customary operation on the touch display 20,to a special, small gesture. As a result of the special definition ofthe finger troughs 22.1 . . . 22.n and 24.1 . . . 24.n in the framesections 36, 38 of the display frame, the finger can slide in acontinuation of the finger troughs onto the touch display, initiating afunction. If the operator notices that he initiated an unintendedinitiation of a function, he can suppress the initiation of the functionby withdrawing his finger into the original position.

As soon as the operator touches the virtual operating element 22.n withhis finger, e.g., starting from the finger trough 40.n, thecorresponding coordinates of the contact point on the touch display aredetected by the control unit 30. The associated function is not releasedin accordance with the definition of a preset gesture until the fingerof the operator leaves a given coordinate area or reaches a predefinedcoordinate area. When the virtual operating element is deflected and istherefore ready for initiation (initiation takes place when the fingerlets go), this is characterized on the operating element by an opticalrecognition, e.g., by a colored edging. If an unintended deflection iscancelled again in that the operating element is withdrawn back into itsorigin, this is made known by a disappearance of this opticalrecognition.

According to another characteristic inventive concept of the inventionvirtual operating elements 48 placed, e.g., in the frame corner 46 ofthe display frame 34 are linked to a special gesture control. They canbe shifted, e.g., along the frame sections 50, 52 in two directions 62,64, as is shown in FIG. 3. Each direction of movement 62, 64 isassociated with a selectable function. Therefore, it is, e.g., possibleto activate a function “A” when moving along the frame section 52 and afunction “B” when moving along the frame section 50. The degree ofdeflection is evaluated and two possibilities of evaluation areprovided.

According to a first possibility of evaluation the degree of thedeflection is immediately transmitted as an analog parameter such as theindication of speed to the function. If the finger is let loose in theextended position, the analogous value immediately jumps to zero. If thefinger is guided in a sliding manner back into the initial position 66,the parameter is set back to zero again analogous to the deflection.This function can be used, e.g., in order to start a movement program ina positive or negative direction and to vary the speed in a graduatedmanner at this time.

A second possibility of evaluation provides that when a definablethreshold value is exceeded, a switching function is initiated. Theactivation of the function does not take place until the finger in theextended position leaves the touch display 20 However, if the finger iswithdrawn back into the zero position without letting loose of the framesections 50, 52, the initiation of the function is prevented.

Another characteristic inventive concept of the invention relates to therealization of a so-called override function (speed regulator) that isrealized by the virtual sliding operating element 60 shown in FIG. 4.For this, the sliding operating element 60 is placed adjacent to theframe section 56 and centrally to the haptic mark 43. The position ofthe sliding operating element 60 can therefore be “detected” and beadjusted by shifting the finger along the frame section 56. The blindadjustment is additionally supported by the haptic marks 42.1 . . .42.n, 43, 54 of the frame section 38, 56. The so-called override can beadjusted by a shifting between two haptic marks by a defined amount,e.g., 20%. Even other analogous magnitudes such as process parameterscan be adjusted in blind operation by the slide operating element 60arranged on the edge side.

Another characteristic inventive feature relates to the symmetricarrangement of the haptic marks 22.1 . . . 22.n, 24.1 . . . 24.nrelative to the longitudinal- and/or transversal central axis of thetouch display 20. The longitudinal central axis is the straight linerunning centrally and parallel to the operating frame shanks of thedisplay frame 34. The transversal central axis runs vertically to it,therefore, centrally between and parallel to the shorter transversalshanks of the display frame 34. As a result, it is ensured that themanual device 10 is suited for a right-hand operation and also aleft-hand operation. This is achieved in particular by the consequentkeyless design of the graphical operator interface and by the symmetricarrangement of the haptic marks.. Therefore, the graphic operatorinterface can be switched by a simple setup function from right-handoperation to left-hand operation. All positions of the virtual operatingelements 22.1 . . . 22.n, 24.1 . . . 24.n are mirrored here on thelongitudinal central axis of the touch display 20.

FIG. 5 shows a back side 66 of the housing 32. Holding strips 70, 72 arearranged on the back side 66 and symmetrically to the longitudinalcentral axis 68 and the manual device 10 can be securely held by thesestrips with one or both hands. The holding strips 70, 72 can have anouter geometry that corresponds to cylindrical sections, wherein theholding strips 70, 72 should emanate from the outer edge, that is, fromthe longitudinal edges of the display frame 34. An enabling switch 74,76 or enabling feeler is integrated in each holding strip 70, 72, one ofwhich must selectively be activated for freeing the movement of theindustrial robot.

This symmetrical arrangement prevents the hands from becoming tiredsince the enabling switches 74, 76 can be alternately activated with theleft or the right hand. If one hand becomes tired, the other hand canassume the enabling without the freeing of the movement beinginterrupted for the movement of the robot.

Another characteristic inventive embodiment of the invention isdistinguished in that a previously customary key switch for theselection of the robot operating types “adjust”, “automatic”, “automatictest” is replaced by a software function. The touch display 20 isbasically a single-channel and therefore unsafe device. A safefunctionality of the software is ensured with the aid of a safetycontrol 78, called safety controller 78 in the following, integrated inthe robot control 16 according to FIG. 6. The safety controller 78 isdescribed in the European patent application 1 035 953, whose disclosureis included in the present application to its full extent. However, theteaching of the invention is not limited to a safety control accordingto European patent application 1 035 953.

Different operating type options in the form of virtual operatingsurfaces 80, 82, 84 such as soft keys are offered for selection on thetouch display 20 by the operating surface 18 as is shown in FIG. 7. Theoperator selects a new operating type “X” by touching one of these softkeys 80, 82, 84. The newly selected operating type is transmitted as thecommand “request new operating type X” to the safety controller 78 bythe software of the operating surface. The safety controller 78 takesgraphical information such as icon 88 corresponding to this operatingtype from its memory 86 and places it on a randomly determined displayposition in a larger image 90. The position of the icon 88 in the image90 is known only to the safety controller 78. This image 90 istransmitted as an image file such as a bitmap to the operating surface18 and is displayed there in a defined position, as is shown in FIG. 8.

The operator must corroborate the operating type recognized by thesafety controller 78 by a fingertip on the icon 88 shown. A touchingposition on the touch display is detected in the form of touchcoordinates and transmitted back to the safety controller 78. The lattercompares the touch position with the random display position of the icon88 in the image 90 that is only known to the safety controller. Thecomparison takes place considering the known position of the image 90 onthe touch display 20. If the touch position (within a defined tolerance)is equal to the display position, the initiated change of operatingtypes is carried out. Otherwise, the change of operating type isrejected and the previous operating type is retained.

This method produces a safe active circuit between the operator andsafety controller 78:

-   -   Operator selects an operating type,    -   Safety controller 78 displays the recognized operating type on        the operating device 10,    -   Operator corroborates the correctness of the displayed operating        type to the safety controller 78,    -   Safety controller 78 sets the new operating type.

As an alternative to the above-cited method, safety controller 78 candisplay an iconized numerical code that must be recognized by theoperator and entered as a number via a displayed keyboard. The touchposition of the displayed numbers of the keyboard is transmitted to thesafety controller that checks the correctness of the input.

The icons 80, 82, 84 are stored with secure technology in the safetycontroller 78.

A request for the change of operating type can optionally also come viaa hardware key switch.

The insertion/withdrawal of the key into/from the operating typeselection switch is imitated by a login/logout method by a PIN.

The possibility after, the touching of the touch display 20, of“pulling” the finger more or less is used according to a characteristicinventive method to generate an analogous movement setting for theindustrial robot 12. In this manner the industrial robot 12 can besensitively controlled according to FIG. 6 in 6 degrees of freedom,e.g., X, Y, Z and orientations A, B, C of a tool 91.

It is possible with the deflection of the finger to give a positionsetting to the industrial robot 12 in a manner similar to the cursorcontrol by touchpad in a notebook. The industrial robot 12 can be movedin this case simultaneously in two coordinate directions, e.g., the Xand Y direction of the Cartesian coordinate system.

In another mode a speed setting for the industrial robot 12 is producedby a deflection of the finger; the more the finger is deflected, themore rapidly the robot moves.

After having touched a selected moving surface 100 shown in FIG. 9 theuser actuates a virtual operating element 92 that is positioned in thearea of the touching point. The virtual operating element 92 cansubsequently be drawn by the finger over the entire touch display 20,even over a boundary 94, and thus generate moving settings. Afterletting loose, the industrial robot the industrial robot 2 immediatelyremains standing. Then, the desired surface 92 must be hit again for anew moving setting.

The sensitivity of the reaction to a finger movement can be continuouslyadjusted by a virtual operating element 96 such as a slide regulator(override) for the position setting and also for the speed setting.

The sensitive surface 100 for the 2-D moving is placed in the vicinityof the display edge 36 so that it can still be readily reached with aclear distance to the display edge by the finger (e.g., with thespread-out thumb).

In order to also be able to move in a third coordinate direction (e.g.,the Z coordinate of the Cartesian coordinate system), a field 101 thatis approximately wide as a finger is laterally placed directly on thedisplay edge 36 with the virtual operating element 98 so that this field101 can be “felt” with the finger in that the finger, in particular thethumb, is guided along the display edge 36. This field produces aone-dimensional moving setting, e.g., in the Z direction.

As a result of the special arrangement the operator can clearlydifferentiate and blindly reach the two moving fields 100, 101: Thefield 101 directly on the display edge 36, wherein the finger hasperceptible contact with the housing edge, actuates the moving settingfor the third dimension (Z dimension). The field 100, that is placedapproximately one finger width or thumb width adjacent to the displayedge 36, actuates the simultaneous moving setting in two dimensions (X-Ydimension).

The industrial robot 12 has 6 degrees of freedom. For the adjusting ofthe orientation of the tool 91 with the three angles (A, B, C) the samemethod as was described above is used. To this end the display isdivided into two zones. The moving fields 100, 101 for the dimensions1-3 (e.g., X, Y, Z) are located, e.g., in the upper zone. The movingfields 102, 103 for the dimensions 4-6, e.g., A, B, C are located, e.g.,in the lower zone. The two fields 100, 101 and 102, 103 can bedistinguished blindly by the haptic marks in the form of nubs 108, 110,112.

After having touched the moving surface 102 the user actuates a virtualoperating element 104 that is positioned in the area of the touchingpoint. Subsequently, the virtual operating element 104 can be shifted onthe touch display in order to produce a moving setting.

Therefore, the industrial robot 12 can be simultaneously moved in all 6degrees of freedom with a multi-touch display 20. With a single-touchdisplay the moving functions can be used only sequentially.

During the moving by the previously explained touch motion function thetouch display is ideally aligned collinearly to the coordinate system ofthe industrial robot. In this case the robot movement optimallycoincides with the finger movement on the touch display.

However, if the user turns away to the side with the operating device10, then this coincidence is no longer given. The direction of movementof the robot then no longer coincides with the direction of movement ofthe finger.

In this case the coordinate system of the touch display must berecalibrated again with the coordinate system of the robot.

According to a characteristic inventive embodiment of the invention aspecial virtual operating element 114 with an indicator 116 is providedto this end on the touch display 20.

The operating element 114 must at first be touched with a finger andsubsequently the finger must be drawn with permanent contact with thetouch display parallel to a selected direction of the robot coordinatesystem, e.g., the X direction. For the visual support of the operatorthe selected direction of the robot coordinate system, e.g., the Xdirection, can be characterized, e.g., by a marking of the bottomsurface. Such a marking is characterized by way of example in FIG. 6 by“200”. After the finger has been lifted off the touch display 20, avector direction is calculated between the first touch point and theletting loose point and the indicator 116 is represented in accordancewith the determined vector direction on the display.

The recalibration method used here is based on the assumption that in abasic state of the calibration, e.g., the longer display edge, thatsets, e.g., the X axis of the display coordinate system, is alignedparallel to a main axis of the robot coordinate system, e.g., the Xaxis. In this “merged” state the robot will correctly follow thedirections given by the finger movement in the touch-controlled handmoving.

In the case of a required recalibration the touch display is rotatedthrough a certain angle to the selected direction of the robotcoordinate system, e.g., the X direction, and the indicator 166 isaligned according to the described method for the manual adjustment ofthe indicator parallel to the selected direction of the robot coordinatesystem, e.g., the X direction, and consequently stands in a certain,sought angle to the X axis of the display coordinate system. An angle iscalculated between the indicator direction and the direction of the Xaxis of the display coordinate system and is entered into a rotationmatrix that now transforms all movements of the finger on the touchdisplay before they are transmitted as a moving setting to the robotcontrol. After the recalibration both coordinate systems are againadjusted with each other in a co-linear manner, quasi-merged.

This recalibration method, including the finger movement, lasts onlyfractions of a second and is therefore one of the most rapid manualrecalibration methods. No additional sensors such as magnetometers orgyrometers are required, as a result of which the method can beeconomically realized on every operating device.

This calibration method can also be used for any other coordinatesystems such as, e.g., freely defined frames.

1. A method for operating an industrial robot (12) with an operatingdevice (10), comprising the method step of touching a virtual operatingelement (92, 98, 104, 106) on a touch-sensitive display of a graphicaloperator interface (18) which display (20) is surrounded by a frame(34), wherein when the virtual operating element (92, 98, 104, 106) istouched, a function associated with the operating element is initiated,characterized in that the speed or the stretch of a deflection of afinger of an operating person when touching the virtual operatingelement (92, 98, 104, 106) of the touch display (20) is detected, that acontrol signal is generated from the speed or the stretch of thedeflection which signal is used for setting the speed or the positionfor movements of the industrial robot (12), and that a first methodsurface (100) for a two-dimensional setting of position is defined at adistance from a frame section (36) serving as a haptic mark, and asecond moving surface (101) for a third setting of position of theindustrial robot is defined inside the distance.
 2. The method accordingto claim 1, characterized in that the deflection of the finger on thedefined moving surface (100, 102) or of a virtual operating elementmoved by the finger is proportional to the moving speed and/or thechange of position of the industrial robot (12).
 3. The method accordingto claim 1 or 2, characterized in that the finger, after having touchedthe defined moving surface (100, 102) and having exceeded a boundary(94), is drawn over the entire touch display (20), which producessettings of position and of moving.
 4. The method according to at leastone of the preceding claims, characterized in that the industrial robot(12) comes to a standstill when the operating person lifts his fingeroff the touch display (20).
 5. The method according to at least one ofthe preceding claims, characterized in that the sensibility of areaction to a movement of a finger over a virtual operating element (96)such as a slide regulator is continuously adjusted for the positionsetting and/or the speed setting.
 6. The method according to at leastone of the preceding claims, characterized in that the defined movingsurface (100, 102) for the two-dimensional position setting (X and Ycoordinates and/or A, B direction) are positioned in such a manner onthe touch display (20) with a distance to the display edge (36) that themoving surface (100, 102) can be operated by a finger such as a spreadthumb.
 7. The method according to at least one of the preceding claims,characterized in that during the controlling of the industrial robot(12) in the third coordinate direction (Z coordinate or C direction) thefinger such as the thumb of an operating person is guided by the displayedge (36) and/or a haptic mark or marks (41.1 . . . 41.n; 108, 110, 112)arranged on the display edge.
 8. The method according to at least one ofthe preceding claims, characterized in that a multi-touch display orsingle-touch display is used as a touch display (20).
 9. The methodaccording to at least one of the preceding claims, characterized in thatthe industrial robot (12) can be moved by a multi-touch display (20) in6 degrees of freedom, wherein the adjustment of moving paths in the X, Yand Z direction takes place in a first zone (100) and the adjustment ofan orientation (A, B C) of a tool (91) takes place in a second zone(102) of the touch display.